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Abstract: A longitudinal study at the TUT (Tshwane University of Technology) used citation analysis to 
analyze the reference lists of 480 Master’s and Doctoral (M and D) theses and dissertations submitted at TUT 
between 2004 and 2007. The purpose was to determine what types of information sources M and D students at 
TUT use most, how the patterns of use differ across the 7 faculties of the university, and to what extent the LIS 
(Library and Information Services) keeps or provides access to the journals that are mostly used by M and D 
students. More than 37,000 citations were analyzed over the 4-year period. The study found several similarities 
but also some distinct differences in the use of information sources across the 7 faculties of TUT. It also identified 
more than 60 different information sources used by M and D students. With regard to journal use, the study found 
that out of 3,641 different journals cited, most journals were only cited once over a period of 4 years. However, a 
small percentage of journals were highly and/or frequently cited. 
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1. Introduction 

1.1 Aims 

The TUT (Tshwane University of Technology) started in 2004 with a longitudinal study to obtain information 
that would assist its LIS (Library and Information Services) in making sound collection development decisions. 
The aims of the study were to identify: 

(1) Which types of information sources do Master’s and Doctoral (M and D) students of different faculties 
use most and least, and to what extent does this change from year to year? 

(2) Which journals do M and D students of different faculties use most and least, and to what extent does this 
change from year to year? 

(3) To what extent does the TUT LIS keep or provide access to journals that are mostly used by M and D 
students? 

1.2 Method 

Library science has developed a number of methods for evaluating the use of information resources. Some of 
the more popular methods include: (1) studying circulation statistics and in-house use of information resources; (2) 
surveying user opinions; (3) analyzing interlibrary loan patterns; (4) doing shelf-availability studies; and (5) 
analyzing bibliographic citations. Although it is generally recognized that no one method alone provides a 
complete picture in assisting with collection development decisions, this study used citation analysis as the 
preferred research method to achieve its aims. This method, with its advantages and disadvantages, and how it 
was applied in this study, was described elsewhere by this author (Swanepoel, 2008). It is nevertheless necessary, 
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for the sake of comprehensiveness, to repeat some of that information in this paper. 

Citation analysis is known to be a low-cost method whereby researchers can gather and study citation data in 
an unobtrusive and non-invasive way. Citation analysis is also a flexible method: It can be applied in the 
assessment of data sources of a group of libraries, or a single collection or a library collection supplemented with 
external information sources. It is also flexible in terms of the size of samples, types of citation sources (whether a 
standard list or a specific collection in a library) as well as the manner of citation selection (Citing, 2002). 
Furthermore, it can be used to focus on the type and number of information sources that researchers or library 
users use in a specific discipline over a period of time. In this regard, the research of Peritz and Sor (1990) and 
Allen, Jacobs and Levy (2006) serve as examples. 

Citation analysis is also a well-studied method in university library environments. Not only do citations play 
an important part in the scholarly communications process, but “citations and the composition of bibliographies 
reflect changes in the information-seeking behavior of academics” as well (Naude, Rensleigh & Du Toit, 2005). In 
this regard, theses and dissertations have proved to be particularly appealing to use for assessing library 
collections, because they serve as a convenient source of in-house research. Furthermore, Zipp (1996) found that 
“The most heavily cited journal titles in theses and dissertations can be used as a surrogate for the titles most 
heavily used by faculty in their publications”. This is because the research interests of graduate students often 
reflect the research interests of their faculty advisers. 

While many studies have reported on the uses and usefulness of citation analysis, some writers, including 
Loree (2007), Griscom (1983) and Ching (2002) have acknowledged the limitations of the method. Concerning 
citation analysis of students bibliographies, Sylvia (1998) highlighted several limitations, including the following: 
(1) Researchers are more likely to use information sources to which they have local access; (2) Researchers may 
add citations to increase a manuscript’s length and scholarship; (3) Manuscripts may include citations of marginal 
importance; (4) Researchers may not cite all works used to prepare a manuscript; and (5) Handbooks and 
textbooks often do not receive citations because those sources are taken for granted by students. 

Being aware of the limitations of citation analysis and acknowledging that the method is sensitive to the skill 
with which it is applied, it was nevertheless decided to use citation analysis as method for this research, because it 
has been proved by many as a valid, reliable and practical tool for comparing a library’s holdings to an 
authoritative list for the purpose of evaluating the quality of a library collection. 

1.3 Data collection 

Data were collected from the reference lists of all (480) theses and dissertations submitted by M and D 
students and accepted by TUT from 2004 to 2007. No sample was taken. However, one thesis from the Faculty of 
Science was not included because its extreme number of citations (3,750) would have skewed results. 

A library assistant photocopied the title pages and reference lists of all theses and dissertations and provided 
each data source with a sequential number. The assistant then captured the following data from the title pages on 
an Excel worksheet: author, title, date, faculty, department at the faculty, language of the thesis, and whether it 
was a master’s thesis or a doctoral dissertation. Subsequently, the researcher collected the following data: number 
of citations per information type, number of citations per thesis, number of theses per department, frequency of 
journal titles cited, and cited periodicals owned or provided access to by the library. All the data collected were 
captured on Excel worksheets. 

1.4 Data analysis 

With regard to Aim (1): The different types of information sources used by M and D students in each of the 7 
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faculties were counted and the numbers obtained per type were summarized per annum. The data were also sorted 
and ranked to identify which types of information sources M and D students in different faculties use most and 
least, and to what extent, the pattern changes from year to year. With regard to Aim (2): A list was made of all the 
different journal titles that were cited by M and D students in each faculty. The researcher then counted the 
number of times each title was cited per annum. This enabled him to determine which journals M and D students 
in different faculties use most and least, and to what extent, this changes from year to year. With regard to Aim (3): 
Each journal title cited by M and D students was compared against a list of journal titles that the LIS makes 
available either by keeping those journals in stock or by providing online access to full text versions of the 
journals. This process resulted in identifying to what extent the LIS keeps or provides access to the journals that 
are mostly used by M and D students. 

2. Findings and discussion 

2.1 Types and number of information sources cited 

To put the findings of this study into context, it is necessary to present an overview of the types and number 
of information sources cited by the students under discussion. Although M and D students of TUT used 64 
different types of information sources from 2004 to 2007, it is clear from Table 1 that they largely relied on books 
and journals for research purposes. Together, citations to books and journals add up to almost 70% of all citations. 
Lurthermore, the top 9 sources are typically those that are readily accessible in or through libraries. One should, 
however, not ignore the fact that several other information types play an important role in the research activities of 
M and D students, especially in-house documents, course material, technical publications, user manuals and 
newsletters/bulletins. 

Citation analysis studies usually refer to less cited information sources as “other” information sources. 
Although this study also used the term “other” to group information resources that individually received less than 
one percent of all citations, it went a step further to actually identify those information sources and rank them 
according to the number of times they were cited (see Table 1). In doing so, this study found that M and D students 
used more than 50 types of information sources that are generally regarded as “grey literature”. This indicated that, 
although M and D students predominantly use books and journals, reference librarians and collection developers 
should take cognizance of the wide variety of so-called grey-literature that M and D students use. 

It should be noted that the number of citations to websites was slightly higher than that reported in Table 1. In 
the process of classifying citations into different types of information sources, citations were first classified by 
content and then by form. Lor instance, a citation to a government publication on the web was classified as a 
government publication and not as a website. Only citations to websites that were not clearly identified as either a 
journal on the web or a government publication on the web, etc., were classified as “websites”. However, Ligure 1 
presents a more complete picture of the number of citations to electronic sources; it shows the number of citations 
to all websites regardless of content, as well as citations to electronic databases and CD-ROMs. Regardless of the 
way citations to websites were classified, this study showed that M and D students of TUT still make far less use 
of electronic formats for research purposes than librarians and faculty members generally suspect. However, 
seeing that there was a slow but gradual increase of electronic formats over 4 years, the ratio between citations to 
paper formats versus electronic formats may seem significantly different over the next 4 years. 
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Table 1 Types and number of information sources cited 



Information sources 


Total citations 2004-2007 


Percentage (%) 


Books and chapters in books 


14,617 


39.11 


Journals and magazines 


11,503 


30.77 


Websites and electronic databases 


3,740 


10.00 


Government publications 


1,387 


3.71 


Conference proceedings and papers 


1,158 


3.10 


Reports 


956 


2.56 


Newspapers 


676 


1.81 


Theses and dissertations 


601 


1.61 


Personal communication 


571 


1.53 


Other ” 


2,169 


5.80 


Total 


37,378 


100.00 



Notes: Consisting of unidentified types (234), in-house documents (205), course material (200), technical publications (176), 

operating manuals and user manuals (169), newsletters and bulletins (158), videos (116), standards and test methods (102), policy 
documents (70), planning documents (65), manuscripts and drafts (61), working papers (48), event programmes (47), statistical data 
(46), discussion and position papers (45), software and computer files (38), comic books (33), fact sheets (33), speeches (29), surveys 
and questionnaires (29), patents (27), pamphlets and brochures (25), codes of conduct (19), radio and television programmes (16), 
briefing documents and position papers (16), case studies (16), specifications (14), media releases (13), trade literature (12), maps 
(11), photos (11), rules and regulations (11), prospectuses (9), circulars (8), staff lists (8), logbooks (7), company profiles (6), 
abstracts (5), catalogues (4), kits (4), proposals (4), exhibit captions (3), event invitations (2), microfilm (2), posters (2), graphics (1), 
sound recordings (1), calendars (1), drawings and diagrams (1), instrumental reviews (1), mathematical models (1), memorabilia (1), 
notices (1), simulation tools (1) and testimonials (1). 
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Figure 1 Citations to paper formats versus electronic formats 



Figure 2 shows some similarities but also distinct differences in the use of information sources by M and D 
students in the so-called SET (science, engineering and technology) Faculties, specifically regarding to books, 
journals and electronic formats. The Faculty of Science clearly uses journals most, followed by books and 
electronic formats, conference papers and reports. Compared to all other faculties, this faculty uses journals most 
by far. The Faculty of Engineering also used journals primarily, followed by books, electronic formats, conference 
papers and reports, but its students use journals far less than students from the Faculty of Science. Engineering 
students also use electronic formats and conference papers more than science students. Students in the Faculty of 
Information Communication and Technology (ICT), on the other hand, first and foremost use web sources, with 
books and journals respectively the second and third most used sources. It is also worth noting that ICT students 
were the only students that use websites and electronic databases more than any other faculty’s M and D students. 
Contrary to this is the relative low use that students of the Faculty of Science made of websites and electronic 
databases. Their use of electronic formats was not only the lowest in the SET faculties but also the lowest in all 
faculties. The patterns showed in Figure 2 did not change much over the 4 years of investigation. 



101 



What 37,000 citations can tell 



100% 

90 % 

80 % 

70 % 

60 % 

50 % 

40 % 

30 % 

20 % 

10% 

0 % 



Science Engineering 



ICT 



□ Other 

I Personal communication 

□ Theses & dissertations 

□ Newspapers 

□ Reports 

■ Proceedings & papers 

□ Govt publications 

□ Web & e-databases 

■ Books 

□ Journals 



Figure 2 Types of information sources cited most by M and D students of SET Faculties, 2004-2007 



Figure 3 shows a different pattern from the one in Figure 2. M and D students in the Social Sciences and 
Flumanities Faculties clearly use books most. They also use noticeably fewer journals than students in the SET 
faculties. This is especially noticeable in the Faculty of Arts. In a list of most cited information sources, the use of 
journals in the Faculty of Arts ranked only fourth, next to books, electronic formats and newspapers. It is also clear 
from Figure 3 that the Faculties of Economic Sciences, Humanities and Management Sciences contributed greatly 
to the fact that government publications ranked fourth in the list of most cited information sources (see Table 1). 




Finance 

Figure 3 Types of information sources used by M and D students of SSHFaculties, 2004-2007 



2.2 Journals used by M and D students 

One of the aims of this study was to determine which journals M and D students in different faculties use 
most and least, and to what extent, this changes from year to year. The results produced a surprisingly large 
amount of data. Authors of the 480 theses and dissertations cited altogether 3,610 different journal titles in 2004 to 
2007. The 3,610 journals received 11,533 citations. The average number of citations to individual journal titles 
was 7.49 and the average (mean) citations to journal articles were 23.93 over the 4-year period. 
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2.2.1 Highly cited journals 

Because of the large amount of data involved, this paper will display only a list of the top 10 most cited 
journals. A list of journals, across all 7 faculties, that received 20 citations or more is available from the author. 



Table 3 Ten most cited journals over a 4-year period 



Journal 


2004 


2005 


2006 


2007 


Total 


Harvard Business Review 


9 


73 


3 


24 


109 


International Journal of Pharmaceutics 


20 


52 


25 


5 


102 


Medicine and Science in Sports and Exercise 


5 


51 


25 


0 


81 


Journal of Analytical Atomic Spectrometry 


25 


38 


7 


8 


78 


Journal of Controlled Release 


11 


45 


19 


3 


78 


Journal of Applied Physiology 


2 


67 


4 


0 


73 


Spectrochimica Acta Part B 


11 


39 


10 


7 


67 


South African Journal of Higher Education 


4 


12 


11 


37 


64 


Human Reproduction 


0 


14 


47 


0 


61 


Pharmaceutical Research 


14 


27 


19 


0 


60 



When the researcher listed and ranked the journals cited in each faculty, it was observed that the vast 
majority of journals had only been cited in 1 or 2 of the years concerned. For instance, of the 50 topmost cited 
journals across all 7 faculties, only 24 journals had been cited in the 4 years of investigation. This is probably an 
indication that the majority of journals that M and D students cited were used by a particular student only for a 
particular research topic. This tendency is more noticeable among some of the SSH faculties. For instance, of the 
138 different journals that M and D students of the Faculty of Arts used during the 4 years of investigation, 133 
(96.4%) journals were used only once. Five journals were used in 2 of the 4 years but no journals were used more 
than that. Contrary to this pattern, the M and D students in the SET faculties have shown a more consistent use of 
journals. Such consistent use will enable the LIS to identify a core list of journals that M and D students in those 
faculties frequently consult. 

2.2.2 Journals that received few citations 

Table 4 presents the number of journals that received 4 or less citations. It is noted that the numbers in the 
columns do not overlap. In other words, if a journal was only cited once in 2004, it was not cited again in 2005 or 
in 2006 or in 2007. 



Table 4 Number of journals that received fewer than 5 citations per year 



Journal 


2004 


2005 


2006 


2007 


Total 


Journals cited only once 


488 


999 


665 


819 


2,971 


Journals cited twice 


110 


271 


143 


172 


696 


Journals cited three times 


49 


105 


64 


57 


275 


Journals cited four times 


26 


67 


36 


39 


168 



The majority of journals (2,97 1 or 82.3% of the total) were only cited once over a period of 4 years. Due to 
the fact that many journals were cited only once, the median and the modus for each of the 4 years was only one 
citation per journal title, notwithstanding that the mean was 8.28 in that period. The results in Tables 3 and 4 
confirm results from other citation studies that a small number of journals generate the majority of journals cited, 
and vice versa. 
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2.2.3 The availability of highly cited journals in TUT libraries 

A third aim of this study was to determine to what extent the LIS keeps or provides access to the journals that 
are mostly used by M and D students. The research found that of the 87 journal titles that received 20 citations or 
more, the LIS either subscribed to or provided online full-text access to a total of 58 titles. In addition, it had back 
copies available in some of its libraries of a further 8 titles of which it recently cancelled the subscriptions. 

3. Conclusion 

From the findings of this study, it can be concluded that M and D students of TUT are far greater users of 
paper-based information sources than of electronic information sources. Over time, there was a gradual increase in 
citations to electronic formats but not as much as one would have expected. Citations to electronic formats 
therefore had no real impact on the composition of reference lists. 

With regard to paper-based sources, M and D students of TUT rely to a large extent on books and journals for 
research purposes. This, however, should not detract attention from the fact that several other information types 
play an important role in the research activities of M and D students, especially that of government publications, 
conference proceedings and papers, reports, newspapers, theses and dissertations and personal communication. 
Since many of the lesser used information sources were not part of the LIS collection, it can therefore also be 
concluded that rather than confining their research to what was available in TUT libraries, several M and D 
students made use of sources elsewhere. Even if lesser used information sources do not qualify to be included in 
the LIS collection, reference librarians of TUT should recognise the need for such sources and become proficient 
in searching and retrieving them. 

This study revealed clearly distinguishable patterns between the information sources that M and D students 
in different faculties of TUT use most and least. This indicates that the LIS should consider separate collection 
development strategies for its 7 faculties. 

Across all faculties, the majority of journals were only cited once over a period of 4 years. However, a small 
percentage of journals were highly cited and/or frequently cited. This confirms the outcome of other citation 
studies: A small number of journals generate the majority of journals cited. This confirmation can be used as a 
rationale for cancelling subscriptions of less-used journals or allocation of funds to faculties who consistently 
make high use of specific journals. 

It is reassuring that TUT libraries are to a large extent able to provide most of the journals that are highly 
cited by M and D students. Although TUT libraries own only a few of the most cited journals, they provide web 
access to most of the others. 

This study provided information that was previously unavailable to librarians and academic staff of TUT. It 
revealed what types of information sources M and D students use more frequently than others, how the patterns of 
use differ across subject disciplines and faculties, and it showed to what extent the LIS keeps or provides access to 
the journals that are mostly used by M and D students. The continued replication of this study will enable the LIS 
to determine even more reliable usage trends that will assist it to make informed decisions regarding which 
information resources to make available to M and D students, especially as Internet and electronic technologies 
evolve. A continuation of this research will also assist librarians and academics to better understand the research 
interests of M and D students at TUT. 
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